Systems and methods for full history dynamic network analysis

ABSTRACT

Provided herein are methods and systems for determining a historical state of a dynamic network. The methods may comprise continuously obtaining data associated with a system from a plurality of different data sources; constructing a full history dynamic network (FHDN) of the system using the data; and providing a state of the system for a historical time instance in response to a query of the FHDN for the historical time instance.

CROSS-REFERENCE

This is a continuation application of International Patent ApplicationNo. PCT/US2019/058951, filed Oct. 30, 2019, which claims the benefit ofU.S. Provisional Patent Application No. 62/754,786, filed on Nov. 2,2018, both of which are hereby incorporated by reference in theirentireties.

BACKGROUND

Network science is the study of large-scale and complex networks. Suchnetworks may include computer networks, cyberphysical systems,telecommunication networks, biological networks, cognitive and semanticnetworks, and social networks. In such networks, distinct elements oractors can be represented by nodes (or vertices), and the connectionsbetween the elements or actors can be represented by links (or edges).

Networks can be illustrated by graphs. Several industries and companieshave developed applications of network and graph processing approachesdesigned for static graphical analysis. For instance, Google Maps canplan complex routes over a current snapshot of the country's roadnetworks, and Facebook can represent its social network as a large graphand has developed the GraphQL language for making queries of such graph.These graphs are not “static” in the traditional sense, because nodesand edges can be added and removed over time, and features of a node oredge may change. However, these graphs do not store the full history ofthe dynamic process. For example, Google Maps may be unable toillustrate the precise state of traffic at a specific time (e.g., Apr.23, 2013, at 7:45 pm PT) and region, and Facebook may be unable toillustrate the state of one of its social network graphs, for example,at an arbitrary point from one year ago. While these networks aredynamic, the storage engine is static in that it only shows the currentstate of the network and previous states of the network at certainpredefined time points, and they do not provide the ability to query thegraph at any arbitrary time point.

SUMMARY

Current techniques for determining the historical state of a network maycomprise periodically taking snapshots of a graph of the network, andthen analyzing a sequence of these snapshots to understand thehistorical dynamic behavior. However, this may not be sufficient fordifferent network domains. For example, for Internet-of-Things (IoT)applications such as an energy distribution system, electricity flow maybe dictated by the precise physical connections of the network andunderstanding complex events like cascading faults may require theknowledge of the precise configuration, second by second, of switchesand connectivity in the electrical network.

Provided herein are methods and systems for analyzing and understandingthe full history of dynamic networks. All networks may be, at theircore, dynamic objects, because edges and nodes of a network can beadded, removed, or change state constantly as the network evolves. In apower grid, for instance, physical assets (electrical lines,transformers, etc) are added and removed over time, switches are openedand closed, and each of these changes fundamentally alters the resultingphysical flow properties of the network. In a transportation network,for instance, roads are opened or closed, and traffic patterns on agiven road change rapidly over time. In order to reason effectivelyabout such networks, there is a need for a model that can accuratelycapture how a network changes over time. Additionally, there is a needfor graphical analysis of a dynamic network where queries can be madeabout the precise state of the network at any given instant in time.

In an aspect, a computer-implemented method of determining a historicalstate of a dynamic network comprises continuously obtaining dataassociated with a system from a plurality of different data sources;constructing a full history dynamic network (FHDN) of the system usingthe data, wherein the FHDN comprises (1) a plurality of nodes that arecapable of dynamically changing, (2) a plurality of edges connecting thenodes, the edges being capable of dynamically changing, and (3) timeseries associated with each of the plurality of nodes and edges; andproviding a state of the system for a historical time instance inresponse to a query of the FHDN for said historical time instance. Thetime series may indicate changes in state of the plurality of nodes andthe plurality of edges over time. The FHDN may be constructed withoutrequiring periodic capturing and storing of snapshots of the system atdifferent points in time

In some embodiments, the state of the system comprises a graphical stateof the entire network, in the as-operated state, at the historical timeinstance. In some embodiments, the state of the system comprises agraphical state for a subset of the network at the historical timeinstance. The graphical state of the network or the subset of thenetwork may be an exact graphical, a substantially exact graphicalstate, or an approximate graphical state. In some embodiments, the FHDNis constructed without requiring periodic capturing and storing ofsnapshots of the network at different points in time. In someembodiments, a historical dynamic behavior of the FHDN is determinedwithout analyzing a sequence of snapshots of the network captured atdifferent points in time. In some embodiments, the FHDN permits thequery for a historical time instance to be answered without requiringfull network instantiation at said historical time instance.

In some embodiments, the plurality of nodes and edges comprises (1) allnodes and edges that had previously existed in the network at any giventime instant, and (2) all nodes and edges that are presently existing inthe network. In some embodiments, the time series for a selected node oredge comprises precise times of additions or removals for the selectednode or edge in the network. In some embodiments, the time series arebased on events or changes occurring at the selected node or edge.

In some embodiments, the state of the system at a historical timeinstance is obtained by using a search algorithm that iterates over theplurality of nodes and searches through the time series. In someembodiments, the search algorithm comprises an iterative graph searchalgorithm that is configured to only check the status of selected nodesor edges on an as-needed basis. In some embodiments, the query comprisesan information request about a subset of nodes at a given time instant,and the search algorithm is configured to query the subset of nodes onlyand directly, without querying other unnecessary nodes.

In some embodiments, the method further comprises: utilizing a blockingtechnique to cache entire connected graphical regions of the FHDN inmemory for any given instant in time. In some embodiments, the blockingtechnique comprises standard blocking, token blocking, orattribute-clustering blocking. In some embodiments, the caching of theentire connected graphical regions in memory allows searches to becarried out more quickly compared to conventional network graphingtechniques.

In some embodiments, the use of the FHDN allows memory/storage savingsof several orders of magnitude compared to conventional network graphingtechniques. In some embodiments, the storage requirements can be reducedby at least about one three orders of magnitude compared to conventionalnetwork graphing techniques. In certain embodiments, the storagerequirements can be reduced by more than three orders of magnitude orless than one order of magnitude.

In some embodiments, the system comprises an electrical distributionsystem. In some embodiments, the electrical distribution systemcomprises a plurality of distribution feeders. In some embodiments, thestate of the electrical distribution system comprises graphical statesof the plurality of distribution feeders at the historical timeinstance. In some embodiments, the state of the electrical distributionsystem comprises graphical states of a subset of the distributionfeeders at the historical time instance. In some embodiments, the FHDNpermits the query for a historical time instance to be answered withoutrequiring full network instantiation of the electrical distributionsystem at said historical time instance.

In some embodiments, (1) the plurality of nodes and edges and (2) thetime series are associated with the plurality of distribution feedersand the connected nodes and branches within each feeder. In someembodiments, the time series for a selected node or edge comprisesprecise times of additions or removals for the selected node or edgewithin the network. In some embodiments, the additions or removals ofthe selected edge corresponds to the opening or closing of a breakerswitch within the electrical distribution system, wherein said breakerswitch is associated with the selected edge.

In some embodiments, the query comprises a query of a precise electricalconfiguration of one or more selected distribution feeders at any giventime instant. In some embodiments, a graph search algorithm is utilizedto query the states of the one or more selected distribution feeders atany given time instant, by only searching the nodes and edges that arecontained in the one or more selected distribution feeders.

In some embodiments, the graph search algorithm is configured to onlyquery the status of nodes and edges that are contained in the selecteddistribution feeders on an as-needed basis. In some embodiments, thegraph search algorithm is not configured to query the nodes and edgescontained in other non-selected distribution feeders. In someembodiments, the network comprises 280000 grid nodes, 320000 edges, and1000000 open/close time series events logged over a period of 6 years.In the above embodiments, the FHDN allows queries at any portion of thenetwork at a historical time instance to be answered, while requiringonly 13.4 MB of storage as compared to 2.1 TB using conventionalgraphing techniques. In some embodiments, the system comprises the Billof Materials for any manufacturing company. In some embodiments, thesystem comprises a supply chain distribution network. In someembodiments, the system comprises a social network comprising of aplurality of users.

In another aspect, a system for determining a historical state of adynamic network comprises: a data aggregation component for continuouslyobtaining data associated with a system from a plurality of differentdata sources; and a network graphing component configured to: constructa full history dynamic network (FHDN) of the system using the data,wherein the FHDN comprises (1) a plurality of nodes that are capable ofdynamically changing, (2) a plurality of edges connecting the nodes, theedges being capable of dynamically changing, and (3) time seriesassociated with each of the plurality of nodes and edges; and provide astate of the system for a historical time instance in response to aquery of the FHDN for said historical time instance.

In another aspect, a non-transitory computer readable medium storinginstructions that, when executed by one or more servers, causes the oneor more servers to perform a method comprises: continuously obtainingdata associated with a system from a plurality of different datasources; constructing a full history dynamic network (FHDN) of thesystem using the data, wherein the FHDN comprises (1) a plurality ofnodes that are capable of dynamically changing, (2) a plurality of edgesconnecting the nodes, the edges being capable of dynamically changing,and (3) time series associated with each of the plurality of nodes andedges; and providing a state of the system for a historical timeinstance in response to a query of the FHDN for said historical timeinstance.

Additional aspects and advantages of the present disclosure will becomereadily apparent to those skilled in this art from the followingdetailed description, wherein only illustrative embodiments of thepresent disclosure are shown and described. As will be realized, thepresent disclosure is capable of other and different embodiments, andits several details are capable of modifications in various obviousrespects, all without departing from the disclosure. Accordingly, thedrawings and description are to be regarded as illustrative in nature,and not as restrictive.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.To the extent publications and patents or patent applicationsincorporated by reference contradict the disclosure contained in thespecification, the specification is intended to supersede and/or takeprecedence over any such contradictory material.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings (also “figure” and “FIG.” herein), of which:

FIG. 1 shows an example of a graph of a full history dynamic network(FHDN);

FIG. 2 shows an example of a graph of a dynamic network at time t; and

FIG. 3 shows an example of a plurality of snapshots at different times.

DETAILED DESCRIPTION

While various embodiments of the invention have been shown and describedherein, it will be obvious to those skilled in the art that suchembodiments are provided by way of example only. Numerous variations,changes, and substitutions may occur to those skilled in the art withoutdeparting from the invention. It should be understood that variousalternatives to the embodiments of the invention described herein may beemployed.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as is commonly understood by one of skill in theart to which the claimed subject matter belongs. It is to be understoodthat the foregoing general description and the following detaileddescription are exemplary and explanatory only and are not restrictiveof any subject matter claimed. In this application, the use of thesingular includes the plural unless specifically stated otherwise.

In the present description, any percentage range, ratio range, orinteger range is to be understood to include the value of any integerwithin the recited range and, when appropriate, fractions thereof (suchas one tenth and one hundredth of an integer), unless otherwiseindicated. It should be understood that the terms “a” and “an” as usedherein refer to “one or more” of the enumerated components unlessotherwise indicated or dictated by its context. The use of thealternative (e.g., “or”) should be understood to mean either one, both,or any combination thereof of the alternatives. As used herein, theterms “include” and “comprise” are used synonymously.

The term “about” or “approximately” can mean within an acceptable errorrange for the particular value as determined by one of ordinary skill inthe art, which will depend in part on how the value is measured ordetermined, e.g., the limitations of the measurement system. Forexample, “about” can mean plus or minus 10%, per the practice in theart. Alternatively, “about” can mean a range of plus or minus 20%, plusor minus 10%, plus or minus 5%, or plus or minus 1% of a given value.Where particular values are described in the application and claims,unless otherwise stated the term “about” meaning within an acceptableerror range for the particular value should be assumed. Also, whereranges and/or subranges of values are provided, the ranges and/orsubranges can include the endpoints of the ranges and/or subranges.

INTRODUCTION

Provided herein is a new paradigm for storing and querying full historydynamic networks (FHDNs). The FHDNs can allow users to query the stateof the network at any instant in history. By using the FHDNs, snapshotsof the graph “on demand” can be reconstructed, and the full network at agiven time point can be produced without storing the snapshotsexplicitly in a database. The FHDNs can allow users to answer queriesabout the graph at an instant in time without instantiating the fullgraph at this time. For example, a common query in an electricaldistribution system is to query the precise electrical configuration ofa distribution feeder at a given time. The entire network may comprisemany thousands of distribution feeders, but the FHDN can allow the userto use graph search algorithms to query the state of the interesteddistribution feeders at any instant in time, by only searching the nodesand edges that are in the distribution feeders of interest.

The FHDN may operate by creating a graph comprising all nodes and edgesthat have ever existed in the network at any point in time. Each edgeand node may include a time series that indicates the times of a change(e.g., additions or removals) for that node or edge in the network. Achange may comprise topological changes such as addition or removal ofan edge or node. In some cases, a change may also include a change in anattribute or property (e.g., weight) of an edge or node. In the case ofan energy distribution system, for example, an edge may be added orremoved multiple times, corresponding to a breaker switch opening orclosing. To reconstruct the graph at a given time, the user can iterateover all nodes, search through time series (e.g., using binary searchfor nodes and edges with large numbers of events), and produce a graphthat corresponds to the network at this time. Any analytic that requiressearch over the graph, such as finding connected components, can be runusing iterative graph search algorithms that only check the status ofnodes or branches on an as-needed basis. If an analytic requiresinformation about a small subset of nodes at a given time, the FHDN datastructure can be queried directly to assess the results of this analyticwithout processing any unnecessary nodes. Because large portions of theFHDN are small enough to store in memory, a blocking technique can beemployed to cache entire connected regions of the graph in memory at anygiven time, allowing the user to carry out these searches quickly andoccasionally only requiring loading a portion of the graph from memory.

Computer-Implemented Method

In an aspect, a computer-implemented method of determining a historicalstate of a dynamic network may comprise continuously obtaining dataassociated with a system from a plurality of different data sources;constructing a full history dynamic network (FHDN) of the system usingthe data; and providing a state of the system for a historical timeinstance in response to a query of the FHDN for the historical timeinstance.

The dynamic networks may be networks that vary over time. For dynamicnetworks, network topology may change over times. For example, nodesand/or edges may be formed and removed over time. The dynamic networksmay comprise, for example, local-area networks, mobile ad hoc wirelessnetworks, communication networks, social networks, energy distributionnetworks, web, and transportation networks. The dynamic networks may bedriven by adversarial models, stochastic models, or game-theoreticmodels.

The data sources may include data from sensors or smart devices, such asappliances, smart meters, wearables, monitoring systems, data stores,customer systems, billing systems, financial systems, crowd source data,weather data, social networks, or any other sensor, enterprise system ordata store. Example of smart meters or sensors may include meters orsensors located at a customer site, or meters or sensors located betweencustomers and a generation or source location. For example, customermeters, grid sensors, or any other sensors on an electrical grid mayprovide measurement data or other information to the grid operator. Thesensors may also comprise, but are not limited to, geophone, hydrophone,lace sensor, microphone, seismometer, sound locator, air flow meter, AFRsensors, blind spot monitor, defect detector, hall effect sensor, wheelspeed sensor, airbag sensors, coolant temperature sensor, fuel levelsensor, fuel pressure sensor, light sensor, MAP sensor, oxygen sensor,oil level sensor, breathalyzer, carbon dioxide sensor, carbon monoxidesensor, electrochemical gas sensor, hydrogen sensor, current sensor,daly detector, electroscope, magnetic anomaly detector, MEMS magneticfield sensor, metal detector, radio direction finder, voltage detector,actinometer, air pollution sensor, ceilometer, gas detector, humistor,leaf sensor, rain gauge, rain sensor, snow gauge, soil moisture sensor,stream gauge, tide gauge, mass flow sensor, water meter, cloud chamber,neuron detection, air speed indicator, depth gauge, magnetic compass,turn coordinator, flame detector, photodiode, wavefront sensor,barometer, pressure sensor, level sensor, viscometer, bolometer,colorimeter, thermometer, proximity sensor, reed switch, and biosensor.By incorporating data from a broad array of sources, the system may becapable of performing complex and detailed analyses, enabling greaterbusiness insights. The data sources may include sensors or databases forother industries and systems without limitation.

The data sources may include large sets of sensors, smart devices, orappliances for any type of industry. The data sources may includesystems, nodes, or devices in a computing network or other systems usedby an enterprise, company, customer or client, or other entity. In oneembodiment, the data sources may include a database of customer orcompany information. The data sources may include data stored in anunstructured database or format, such as a Hadoop distributed filesystem (HDFS). The data sources may include data stored by a customersystem, such as a customer information system (CIS), a customerrelationship management (CRM) system, or a call center system. The datasources may include data stored or managed by an enterprise system, suchas a billing system, financial system, supply chain management (SCM)system, asset management system, and/or workforce management system. Thedata sources may include data stored or managed by operational systems,such as a distributed resource management system (DRMS), documentmanagement system (DMS), content management system (CMS), energymanagement system (EMS), geographic information system (GIS),globalization management system (GMS), and/or supervisory control anddata acquisition (SCADA) system. The data sources may include data aboutdevice events. The device events may include, for example, devicefailure, reboot, outage, tamper, and the like. The data sources mayinclude social media data such as data from Facebook®, LinkedIn®,Twitter®, or another social network or social network database. The datasources may also include other external sources such as data fromweather services or websites and/or data from online application programinterfaces (APIs) such as those provided by Google®. The data sourcesmay comprise an external database.

The full history dynamic network (FHDN) may comprise the entire historyof a dynamic network. For instance, the FHDN may comprise the nodes,edges, and the relationship between the nodes and edges of a dynamicnetwork at any time in the history of the dynamic network. If a dynamicnetwork was created in the year 2010, for example, the FHDN may compriseall the edges, all the nodes, and time series that have ever existed inthe dynamic network since the year 2010. Additionally, the FHDN may alsocomprise information on changes of all the edges and all the nodes inthe dynamic network over time. The FHDN may be configured to store andquery the information of a dynamic network. The FHDN may be configuredto query the state of the system at any instant in history of a dynamicnetwork.

The FHDN may comprise (1) a plurality of nodes that are capable ofdynamically changing, (2) a plurality of edges connecting the nodes, theedges being capable of dynamically changing, and (3) time seriesassociated with each of the plurality of nodes and edges.

A given node of the plurality of nodes may be a redistribution point ora communication endpoint. If the network is a physical network, the nodemay be an active electronic device that is attached to a network. Inthis situation, the node may be capable of creating, receiving, ortransmitting information over a communications channel. The physicalnetwork node may be data communication equipment (DCE) such as a modem,hub, bridge or switch, or data terminal equipment (DTE) such as adigital telephone handset, a printer or a host computer. If the networkis a local area network (LAN) or wide area network (WAN), every LAN orWAN node that are at least data link layer devices may a networkaddress, typically one for each network interface controller itpossesses. Examples of nodes in physical network may comprise computers,packet switches, xDSL modems (with Ethernet interface) and wireless LANaccess points. If the network is the Internet or an Intranet, physicalnetwork nodes may be host computers that are identified by an IPaddress.

In a fixed telephone network, a node may be a public or privatetelephone exchange, a remote concentrator or a computer providing someintelligent network service. In cellular communication, examples ofnodes may comprise switching points and databases such as the Basestation controller, Home Location Register, Gateway GPRS Support Node(GGSN) and Serving GPRS Support Node (SGSN). In cable television systems(CATV), the nodes may comprise a fiber optic node. The fiber optic nodemay be homes or businesses within a specific geographic area that areserved from a common fiber optic receiver. If the network is adistributed system, the nodes may be clients, servers or peers.

A given edge of the plurality of edges may be one of the connectionsbetween two nodes (or vertices) of the network. Edges can be directed,meaning they point from one node to another node. In this case, the twonodes connected by the directed edge may be in a one-directionrelationship. The one-direction relationship may be such that one nodesends information to the other node but does not receive any informationfrom the other node. Edges can also be undirected, in which case theyare bidirectional. In this case, the two nodes connected by the directededge may be in a two-direction relationship. The two-directionrelationship may be such that one node sends information to the othernode, and also receives any information from the other node. In somecases, nodes may not send information to each other.

A time series may be a series of data points in time order. A timeseries may be a sequence taken at successive equally-spaced points intime. Time series may comprise sequence of discrete-time data. The timeseries may indicate when nodes or edges change, e.g., when nodes oredges are added, removed, activated, deactivated, or connected ordisconnected from another node. Alternatively or additionally, the timeseries may indicate time-varying properties of the nodes or edges. Inthe case of an electrical distribution network, the nodes may be powerplants, transmission substations, distribution feeders, transformers,breakers, and consumers, for example. The edges may be transmissionlines and other wires that connect such nodes. The time series mayindicate, for example, when a power plant is online or offline or when abreaker is open or closed. The time series may additionally indicate,for example, the power output of a power plant over time. In the case ofa social network, the nodes may be businesses and people that have aprofile on the social network. The edges may be the relationshipsbetween those businesses and people (e.g., friend, follower, etc.). Thetime series may indicate when relationships on the social network beganor ended. The time series may additionally indicate, for example, howproperties of the businesses or people in the social network change overtime (e.g., how a person's relationship status, occupation, or locationchanges over time). In the case of a supply chain distribution network,the nodes may be supplier plants, assembly plants, regional distributioncenters, local distribution centers, and customer locations. The nodesmay be roads, railways, shipping lanes, and flight paths that connectthe nodes in the supply chain network to each other. The time series mayindicate, for example, whether a plant is operational at particulartimes and whether roads or railways are open at particular times.

The time series may represent the dynamically changing nodes and edgesin the FHDN. The changing may be systematic or non-systematic. If thechanging is systematic, the time series may be obtained at a systematictime interval. If the changing is non-systematic, the time series may beobtained at non-systematic time intervals. For instance, the time seriesmay be first obtained at every 1 s for the time period of 1 minute, andthen obtained every 10 s for the time period of 10 hours. In some cases,although the system may be continuously or periodically monitored forchanges to nodes and edges, time series entries may only be stored inmemory when actual changes to the nodes and edges occur. This can reducethe amount of memory required to store the FHDN. A query to the FHDN maycomprise a request for information from the FHDN. The query may compriserequesting the state of the dynamic network at a historical timeinstant. The query can be posed by choosing parameters from a menu,where the database system presents a list of parameters from which theuser can choose. The query can also be posed by query by example, wherethe system presents a blank record and lets the user specify the fieldsand values that define the query. The fields or values may specify thehistorical time instant directly, or they may specify the historicaltime instant indirectly. For example, the fields or values may specify achange that happens to a particular node. The query can be posed byquery language, where user makes requests for information in the form ofa stylized query that must be written in a special query language.

FIG. 1 shows an example of a graph of FHDN. In FIG. 1, the FHDN 100comprises a plurality of nodes 102 that are capable of dynamicallychanging, a plurality of edges 104 connecting the nodes and beingcapable of dynamically changing, and time series associated with each ofthe plurality of nodes 106 and edges 108. The FHDN may comprise all thenodes and edges that have ever existed in the dynamic network. The timeseries 106 of one node may be T1=[A, I, A, A, A, . . . ], whichrepresents, at a specific time interval, the changes of the node overtime are “active, inactive, active, active, active, . . . .” The timeseries 108 of one edge may be T2=[A, I, I, A, A, . . . ], whichrepresents, at a specific time interval, the changes of the edge overtime are “active, inactive, inactive, active, active, . . . .”

FIG. 2 shows an example of a graph of dynamic network at time t. In FIG.2, the dynamic network 200 comprises a plurality of nodes 202 that areactivated at time t (shaded), and a plurality of active edges 204(represented by solid lines) connecting the nodes that are activated attime t. At time t, the rest of the nodes 206 (unshaded) and the rest ofthe edges 208 (represented by dashed) are not activated.

The state of the system may comprise a graphical state of the entirenetwork, in the as-operated state, at the historical time instance. Thehistorical time instance may be any time in the history of the dynamicnetwork defined by the user. For instance, the historical time instancecan be a time point in the history of the dynamic network (e.g., at 10am on Aug. 12, 2000), a time period in the history of the dynamicnetwork (e.g., between 1 am and 12 pm on Oct. 11, 1980), or acombination thereof. A graphical state may comprise a graph structure,which may be a graphical representation of data involving relationships(edges) between nodes. A graph may be an ordered pair comprising a setof nodes together with a set of edges. Nodes may be a set together witha relation of incidence that associates with each edge connecting thetwo nodes.

Graphs can be used to model many types of relations and processes. Forinstance, in computer science, graphs may be used to represent networksof communication, data organization, computational devices, the flow ofcomputation, etc. In one example, the link structure of a website can berepresented by a directed graph, in which the nodes represent web pagesand directed edges represent links from one page to another. Anotherexample is in chemistry, where a graph may make a natural model for amolecule—nodes represent atoms and edges represent bonds. In statisticalphysics, graphs can represent local connections between interactingparts of a system, as well as the dynamics of a physical process on suchsystems. Similarly, in computational neuroscience graphs can be used torepresent functional connections between brain areas that interact togive rise to various cognitive processes, where the nodes representdifferent areas of the brain and the edges represent the connectionsbetween those areas. Graphs may be used to represent the micro-scalechannels of porous media, in which the nodes represent the pores and theedges represent the smaller channels connecting the pores. In biology, anode can represent regions where certain species exist (or inhabit) andthe edges represent migration paths or movement between the regions. Thegraph can also be applied to problems in social media, travel, computerchip design, mapping the progression of neuro-degenerative diseases, andmany other fields. In the case of travel, the systems and methodsdescribed herein can be used to create a FHDN of a travel network (e.g.,roads, waterways, flight paths, etc.). Nodes may represent differentdestinations (e.g., cities) in the travel network, and edges mayrepresent different paths between such destinations. The edges may beweighted by distance or travel time. Time series in the FHDN mayindicate whether particular paths are open at particular times (e.g.,whether particular roads are accessible or whether particular flightsare available). The FHDN of the travel network can be used to determineoptimal routes (e.g., routes that are quickest or shortest) from onedestination to another.

In the case of computer chip design, the system and methods describedherein can be used to create an FHDN of a computer chip, which can beused to analyze component failures over time and subsequently predictfuture failures.

In the case of neuro-degenerative diseases, the systems and methodsdescribed herein can be used to create an FHDN of a human brain. Thenodes of the FHDN may represent neurons in the brain, and the edges maybe connections between the neurons. The time series may indicate whetherparticular neurons have been adversely impacted by the diseaseprogression. Creating FHDNs of patients' brains can aid doctors inpredicting and preventing disease progression in other patients.

The systems and methods described herein can also be used to createFHDNs of oil and gas processing pipelines. An oil and gas processingpipeline may include drilling assets, refining assets, and pipelineassets (e.g., pumps, compressors, heat exchangers, and valves). Thenodes in the FHDN may represent the drilling and refining assets, andthe edges may represent the pipeline assets. Time series may indicatewhether certain assets are operational at certain times, and they mayalso indicate the capacity or output of those assets over time.

A graph structure can be extended by assigning a weight to each edge ofthe graph. Graphs with weights, or weighted graphs, may be used torepresent structures in which pairwise connections have some numericalvalues. For example, if a graph represents a road network, the weightscould represent the length of each road. There may be several weightsassociated with each edge, including distance (as in the previousexample), travel time, or monetary cost. A graph structure can also beextended by assigning time series to each edge and node of the graph.Graphs with time series may be used to represent structures in whichpairwise connections have some values that change over time. Forexample, if a graph represents a road network, the time series couldrepresent the traffic of each road over time. The weights mayalternatively or additionally represent the strength of therelationships or connections between nodes.

The graphs can be stored in a computer system. The data structure usedto store the graphs may depend on both the graph structure and thealgorithm used for manipulating the graph. The data structure maycomprise list structure, matrix structure, or a combination of both.List structures may be used for sparse graphs as they have smallermemory requirements. Matrix structures may provide faster access forsome applications but can consume huge amounts of memory. Different liststructure and matrix structure may comprise adjacency list, adjacencymatrix, and incidence matrix. For the adjacency list, nodes may bestored as records or objects, and every vertex may store a list ofadjacent vertices. This data structure may allow the storage ofadditional data on the nodes. For the adjacency matrix, atwo-dimensional matrix may be used, in which the rows represent sourcenodes and columns represent destination nodes, and data on edges andnodes may be stored externally. For the incidence matrix, atwo-dimensional Boolean matrix may be used, in which the rows representthe nodes and columns represent the edges, and the entries may indicatewhether the vertex at a row is incident to the edge at a column. Theentry may be a time series that indicates whether the vertex at a row isincident to the edge at a column at any given time instance.

The provided FHDN may be used as a graph structure for a graph database.Data objects may be stored in the graph database in the form of FHDN. Agraph database may be a database that uses graph structures for querieswith nodes, edges, and properties to represent and store data. In oneembodiment, every element contains a direct pointer to its adjacentelements and no index lookups are necessary. A key concept of the systemmay be the graph (or edge or relationship), which directly relates dataitems in the store. The relationships may allow data in the store to belinked together directly, and in many cases retrieved with oneoperation. The pointers may be unidirectional or bidirectional.

Graph databases may allow simple and fast retrieval of complexhierarchical structures that are difficult to model in relationalsystems. The storage mechanism of graph databases may comprise amechanism that depends on a relational engine and “store” the graph datain a table, or a mechanism that uses a key-value store ordocument-oriented database for storage, making them inherently NoSQLstructures. Some graph databases based on non-relational storage enginesmay also add the concept of tags or properties, which are essentiallyrelationships having a pointer to another document. Retrieving data froma graph database may require a query language. Some graph databases maybe accessed through application programming interfaces (APIs).

Graph databases may employ a graph structure comprising nodes, edges,and properties. The graph structure may be a FHDN as described elsewhereherein. For example, a time series indicating times of change of a nodeor edge may be recorded with the node or edge. Nodes may represententities such as people, businesses, accounts, or any other item to betracked. Edges may be the lines that connect nodes to other nodes, andthey may represent the relationship between them. Edges may represent anabstraction that is not directly implemented in the systems. Propertiesmay be metadata or data about a node. For example, in the case of asocial network in which people are nodes, the properties may includedemographic or personal information about the people (e.g., age, sex,employer, school, location, etc.). The method disclosed herein maycomprise external graph databases. External graph databases may compriseAllegroGraph, AnzoGraph, ArangoDB, DataStax, InfiniteGraph, Marklogic,Microsoft SQL server, Neo4j, OpenLink Virtuoso, Oracle Spatial andGraph, OrientDB, SAP HANA, Sparksee, Sqrrl Enterprise, Teradata Aster,or other similar types of databases.

The state of the system may comprise a graphical state for a subset ofthe network at the historical time instance. The graphical state maycomprise all the nodes and edges that are requested by a query at thehistorical time instance defined by the user of the dynamic network. Allthe nodes and edges that are requested by a query at the historical timeinstance defined by the user may be a subset of the network. All thenodes and edges that are requested by a query at the historical timeinstance defined by the user may be the full network. The subset of thenetwork may comprise a subset of nodes, a subset of edges, and/or asubset of time series of the full network. The subset of the network maycomprise at least about 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%,50%, 60%, 70%, 80%, 90% or greater of nodes of the full network. Inother cases, the subset of the network may comprise at most about 90%,80%, 70%, 60%, 50%, 40%, 30%, 20%, 15%, 10%, 9%, 8% or less of nodes ofthe full network. The subset of the network may comprise at least about4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%or greater of edges of the full network. In other cases, the subset ofthe network may comprise at most about 90%, 80%, 70%, 60%, 50%, 40%,30%, 20%, 15%, 10%, 9%, 8% or less of edges of the full network. Thesubset of the network may comprise at least about 4%, 5%, 6%, 7%, 8%,9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% or greater of timeseries of the full network. In other cases, the subset of the networkmay comprise at most about 90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, 15%,10%, 9%, 8% or less of time series of the full network.

The FHDN may be constructed without requiring periodic capturing andstoring of snapshots of the network at different points in time.

A snapshot may comprise the state of a system at a particular point intime. FIG. 3 shows an example of a plurality of snapshots of a dynamicnetwork at different times. In FIG. 3, a time-series of snapshots of adynamic network for time instants, t=1 to t=6 are shown. The dotted edgeindicates that there is no relationship, and a solid edge indicates arelationship. For example, (1, 2) have no relationship at t=1, and (2,4) have no relationship at t=6.

A historical dynamic behavior of the FHDN may be determined withoutanalyzing a sequence of snapshots of the network captured at differentpoints in time. The historical dynamic behavior may comprise the changesof the FHDN at a historical time instance defined by the user. Thehistorical dynamic behavior may comprise the changes of the FHDN over aperiod of time defined by the user. To determine the historical dynamicbehavior of the FHDN without analyzing a sequence of snapshots of thenetwork captured at different points in time, time series may beobtained and used with information regarding nodes and edges in thedynamic network. In this situation, no duplication of information aboutnodes and edges across snapshots may be required.

The FHDN may permit the query for a historical time instance to beanswered without requiring full network instantiation at the historicaltime instance. Instantiation may be the creation of a real instance orparticular realization of an abstraction or template such as a class ofobjects or a computer process. To answer the query for a historical timestance without requiring full network instantiation at the historicaltime instance, time series may be obtained and used with informationregarding nodes and edges in the dynamic network. In this situation,only the time series, nodes and edges relevant to the query may besearched and obtained, and the time series, nodes and edges irrelevantto the query need not be searched/processed. In some cases, particulartime series, nodes, and edges may be inferred from other time series,nodes, and edges. For example, the state of a respective node can beinferred from nodes and edges that are adjacent to the respective node.It may be desirable to infer the state of a node when data about thatnode is missing, e.g., due to a malfunctioning sensor or otherconnection issue.

The plurality of nodes and edges may comprise (1) all nodes and edgesthat have previously existed in the network at a given time instantafter data started being collected about the network, and (2) all nodesand edges that are presently existing in the network. In otherembodiments, the plurality of nodes and edges comprises (1) all nodesand edges that had previously existed in the network at any given timeinstant, (2) all nodes and edges that presently exist in the network,and (3) all nodes and edges that will be existing in the network. Thenumber of all nodes may be at least 1, 10, 50, 100, 200, 300, 400, 500,1000, 10000, 100000, 1000000, 10000000, 100000000, 1000000000, orgreater. The number of all nodes may be at most 1000000000, 100000000,10000000, 1000000, 100000, 10000, 1000, 500, 400, 300, 200, 100, 50, 10or less. The number of edges may be comparable to the number of nodes,or the number of edges may be greater than the number of nodes. In somecases, the number of edges may be less than the number of nodes.

The time series may be based on events or changes occurring at theselected node or edge. In some cases, the events or changes may includea topology change. Alternatively or additionally, the change may includea change in a property or attribute (e.g., weight, directionality) of anedge or node. The events or changes may comprise selecting a node,unselecting a node, selecting an edge, and unselecting an edge. The timeseries for a selected node or edge may comprise precise times ofadditions or removals for the selected node or edge in the network. Thetime series for a selected node or edge may comprise precise times ofadditions, removals, and amendments for the selected node or edge in thenetwork. The time series may be obtained in systematic or non-systematictime intervals. The systematic time interval may be at least every 0.1microsecond (μs), 1 μs, 10 μs, 100 μs, 1 millisecond (ms), 10 ms, 100ms, 1 second (s), 2 s, 3 s, 10 s, 30 s, 60 s, 2 minutes (m), 3 m, 4 m, 5m, 10 m, or greater. In some embodiments, the systematic time intervalmay be at most every 10 m, 5 m, 4 m, 3 m, 2 m, 1 m, 30 s, 10 s, 3 s, 2s, 1 s, 100 ms, 10 ms, 1 ms, 100 μs, 10 μs, 1 μs, 1 μs, or less. For thenon-systematic time interval, the time series may be first obtained at afirst time for a first time period, and then obtained at a second timefor a second time period. For instance, the time series may be firstobtained at every 1 s for a first time period of 1 minute, and thenobtained every 10 s for a second time period of 10 hours.

The time series may be stored in a storage, including a high-throughput,distributed key-value data store. The distributed key/value store mayprovide reliability and scalability with an ability to store massivevolumes of datasets and operate with high reliability. The key/valuestore may also be optimized with tight control over tradeoffs betweenavailability, consistency, and cost-effectiveness. The data persistenceprocess may be designed to take advantage of elastic computer nodes andscale-out should additional processing be required to keep up with thearrival rate of messages onto the distributed queues. The storage mayinclude a wide variety of database types. For example, distributedkey-value data stores may be ideal for handling time-series and otherunstructured data. The key-value data stores may be designed to handlelarge amounts of data across many commodity servers, may provide highavailability with no single point of failure. Relational data store maybe used to store and query business types with complex entityrelationships. Multi-dimensional data stores may be used to store andaccess aggregates including aggregated data that is from a plurality ofdifferent data sources or data stores.

The state of the system at a historical time instance may be obtained byusing a search algorithm that iterates over the plurality of nodes andsearches through the time series. The number of the plurality of nodesmay not be all the nodes that have ever existed in a dynamic network. Inother cases, the number of the plurality of nodes may be all the nodesthat have ever existed in a dynamic network.

The state of the system at a historical time instance may be obtained byusing a search algorithm that iterates over the plurality of edges andsearches through the time series. The number of the plurality of edgesmay not be all the edges that have ever existed in a dynamic network. Inother cases, the number of the plurality of edge may be all the edgesthat have ever existed in a dynamic network.

The search algorithm may be any algorithm which solves the searchproblem to retrieve information stored within the data structure orcalculated in the search space of a problem domain. Examples of searchalgorithms may include but are not limited to a linked list, an arraydata structure, or a search tree. The appropriate search algorithm maydepend on the data structure being searched and prior knowledge aboutthe data. Searching may encompass algorithms that query the datastructure, such as the SQL SELECT command.

Search algorithms can comprise linear search algorithms, binary searchalgorithms, jump search algorithms, interpolation search algorithms,exponential search algorithms, sublist search algorithm, comparisonsearch algorithms, and digital search algorithms. Linear searchalgorithms may check every record for the one associated with a targetkey in a linear fashion. Binary search algorithm may repeatedly targetthe center of the search structure and divide the search space in half.Comparison search algorithms may improve on linear searching bysuccessively eliminating records based on comparisons of the keys untilthe target record is found. Comparison search algorithms can be appliedon data structures with a defined order. Digital search algorithms maywork based on the properties of digits in data structures that usenumerical keys.

The search algorithm may comprise an iterative graph search algorithmthat is configured to only check the status of selected nodes or edgeson an as-needed basis. The graph search algorithms may specify an orderto search through the nodes of a graph. The graph search algorithm maybe, for example, a connected component search such as depth-first searchor breadth-first search. For instance, the graph search algorithms maystart at the source node and keep searching until the target node isfound, then the frontier may comprise nodes that haven't been exploredyet, and each iteration, a node may be taken off the frontier, and addits neighbors to the frontier. The as-needed basis can be set up by theusers. For example, if an analytic requires information about a smallsubset of nodes at a given time, the FHDN data structure can be querieddirectly to assess the results of this analytic without touching anyunnecessary nodes. The unnecessary nodes may be the nodes that are notrelated to the queries. The unnecessary nodes may comprise at leastabout: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%,80%, 90% or greater of all the nodes in the FHDN. In other cases, theunnecessary nodes may comprise at most about 90%, 80%, 70%, 60%, 50%,40%, 30%, 20%, 15%, 10%, 9%, 8% or less of all the nodes in the FHDN.

The query may comprise an information request about a subset of nodes ata given time instant. The query may comprise an information requestabout a subset of edges at a given time instance. The query may comprisean information request about a subset of nodes and edges at a given timeinstance. The query may comprise an information request about a subsetof nodes at multiple given time instants. The query may comprise aninformation request about a subset of edges at multiple given timeinstants. The query may comprise an information request about a subsetof nodes and edges at multiple given time instants.

The search algorithm may be configured to query the subset of nodes onlyand directly, without querying other unnecessary nodes. This can enablequicker response times and more targeted/focused queries. In some cases,the subset of nodes/edges or nodes/edges selected to be queried forreproducing a graph may be the nodes/edges with frequent events changes.For instance, nodes or edges with the number of changes above athreshold may be queried. In another example, nodes or edges withgreater number of changes are queried prior to nodes or edges withsmaller number of changes. The search algorithm may be configured toquery the subset of nodes only and indirectly, without querying otherunnecessary nodes. The search algorithm may be configured to query thesubset of nodes only and directly, without querying other unnecessaryedges. The search algorithm may be configured to query the subset ofnodes only and indirectly, without querying other unnecessary edges. Thesearch algorithm may be configured to query the subset of nodes only anddirectly, without querying other unnecessary nodes and edges. The searchalgorithm may be configured to query the subset of nodes only andindirectly, without querying other unnecessary nodes and edges. Theunnecessary nodes and edges may be the nodes and edges that are notrelated to the queries. The unnecessary nodes may comprise at leastabout: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%,80%, 90% or greater of all the nodes in the FHDN. In other cases, theunnecessary nodes may comprise at most about 90%, 80%, 70%, 60%, 50%,40%, 30%, 20%, 15%, 10%, 9%, 8% or less of all the nodes in the FHDN.The unnecessary edges may comprise at least about: 4%, 5%, 6%, 7%, 8%,9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% or greater of allthe edges in the FHDN. In other cases, the unnecessary edges maycomprise at most about 90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, 15%, 10%,9%, 8% or less of all the edges in the FHDN.

The method may further comprise utilizing a blocking technique to cacheone or more connected graphical regions of the FHDN in memory for anygiven instant in time. The memory may be volatile RAM or non-volatilememory. Volatile RAM may be implemented as dynamic RAM (DRAM), whichrequires power continually in order to refresh or maintain the data inthe memory. Non-volatile memory may be a magnetic hard drive, a magneticoptical drive, an optical drive (e.g., a DVD RAM), or other type ofmemory system that maintains data even after power is removed from thesystem. The non-volatile memory may also be a random-access memory. Thenon-volatile memory can be a local device coupled directly to the restof the components in the data processing system. A non-volatile memorythat is remote from the system, such as a network storage device coupledto any of the computer systems described herein through a networkinterface such as a modem or Ethernet interface, can also be used.

The blocking technique may comprise standard blocking, token blocking,or attribute-clustering blocking. The blocking technique may help avoidmemory bandwidth bottlenecks in a number of applications. The blockingtechnique may exploit the inherent data reuse available in theapplication by ensuring that data remains in cache across multiple uses.The blocking technique can be performed on 1-D, 2-D or 3-D spatial datastructures. Some iterative applications can further benefit fromblocking over multiple iterations to further mitigate bandwidthbottlenecks. The blocking technique may comprise a combination of loopsplitting and interchange. Cache blocking may be a technique torearrange data access to pull subsets (blocks) of data into cache and tooperate on this block to avoid having to repeatedly fetch data from mainmemory.

The caching of one or more connected graphical regions in memory mayallow searches to be carried out more quickly compared to conventionalnetwork graphing techniques. A connected graphical region may refer to aconnected region of a graph which contains no unreachablevertices/nodes. For instance, a connected region of a graph may beloaded into cache and remained across multiple uses. The caching of theone or more connected graphical regions in memory may allow searches tobe carried out 5-65 times faster, or more, compared to conventionalnetwork graphing techniques.

The use of the FHDN may allow memory/storage savings of several ordersof magnitude compared to conventional network graphing techniques. Thestorage requirements can be reduced by at least three orders ofmagnitude compared to conventional network graphing techniques. Thestorage requirements can be reduced by from 1 to 10, 2 to 9, 3 to 8, 4to 7, 5 to 6 orders of magnitude compared to conventional networkgraphing techniques. The storage requirements can be reduced by at least3, 4, 5, 6, 7, 8, 9, 10 or more orders of magnitude compared toconventional network graphing techniques. In some cases, the storagerequirements can be reduced by at least 10, 9, 8, 7, 6, 5, 4 or lessorders of magnitude compared to conventional network graphingtechniques.

Electrical Distribution System

The system may comprise an electrical distribution system. Theelectrical distribution system may comprise different arrangements. Thearrangements may comprise a radial system, an expanded radial system, aradial system with primary selectivity, a primary and secondary simpleradial system, a primary loop system, a secondary selective system, aprimary selective system, a sparing transformer system, a secondary spotnetwork, and a composite system.

The electrical distribution system may comprise a plurality ofdistribution feeders. The plurality of distribution feeders may beconnected to each other through a plurality of connections. Thedistribution feeders may comprise a plurality of nodes. The nodes maycomprise an electrical consumption device and an electrical grid. Thedistribution feeders may comprise a plurality of edges. The edges maycomprise electrical wires connecting the nodes.

The state of the electrical distribution system may comprise graphicalstates of the plurality of distribution feeders at the historical timeinstance. The state of the electrical distribution system may comprisegraphical states of the plurality of connections connecting distributionfeeders at the historical time instance.

The state of the electrical distribution system may comprise graphicalstates of a subset of the distribution feeders at the historical timeinstance. The graphical state may comprise all the distribution feedersthat are requested by a query at the historical time instance. All thedistribution feeders at the historical time instance may be a subset ofthe electrical distribution system. The subset of the distributionfeeders may comprise at least about: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%,20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% or greater of distributionfeeders of the electricity distribution system. In other cases, thesubset of the distribution feeders may comprise at most about 90%, 80%,70%, 60%, 50%, 40%, 30%, 20%, 15%, 10%, 9%, 8% or less of thedistribution feeders of the electricity distribution system.

The FHDN may permit the query for a historical time instance to beanswered without requiring full network instantiation of the electricaldistribution system at the historical time instance. Instantiation maybe the creation of a real instance or particular realization of anabstraction or template such as a class of data objects or a computerprocess.

The plurality of nodes and edges and the time series may be associatedwith the plurality of distribution feeders and the connected nodes andbranches within each feeder. The plurality of nodes and edges maycomprise (1) all nodes and edges that had previously existed in thedistribution feeders at any given time instant, and (2) all nodes andedges that are presently existing in the distribution feeders. In otherembodiments, the plurality of nodes and edges comprises (1) all nodesand edges that had previously existed in the distribution feeders at anygiven time instant, (2) all nodes and edges that are presently existingin the distribution feeders, and (3) all nodes and edges that will beexisting in the distribution feeders.

The time series may comprise precise times of additions or removals forthe selected node or edge in the distribution feeders. The time seriesmay comprise precise times of additions, removals, or amendments for theselected node or edge in the distribution feeders. The time series maybe obtained through systematic or non-systematic time intervals. For thenon-systematic time interval, the time series may be first obtained at afirst time for a first time period, and then obtained at a second timefor a second time period. For instance, the time series may be firstobtained at every 1 s for a first time period of 1 minute, and thenobtained every 10 s for a second time period of 10 hours. The additionsor removals of the selected edge may correspond to the opening orclosing of a breaker switch within the electrical distribution system,wherein the breaker switch is associated with the selected edge. Theamendments of the selected edges may represent the changes of electricalconsumption of the selected edges.

The query may comprise a query of a precise electrical configuration ofone or more selected distribution feeders at any given time instant.

The graph search algorithm may be utilized to query the states of theone or more selected distribution feeders at any given time instant, byonly searching the nodes and edges that are contained in the one or moreselected distribution feeders. The search algorithm may be any algorithmwhich solves the search problem to retrieve information stored withinsome data structure or calculated in the search space of a problemdomain. Examples of search algorithms include, but are not limited to, alinked list, an array data structure, or a search tree. Searchalgorithms can also comprise linear search algorithms, binary searchalgorithms, jump search algorithms, interpolation search algorithms,exponential search algorithms, sublist search algorithm, comparisonsearch algorithms, and digital search algorithms.

The graph search algorithm may not be configured to query the nodes andedges contained in other non-selected distribution feeders. The searchalgorithm may be configured to query the subset of nodes only anddirectly, without querying other unnecessary nodes. The search algorithmmay be configured to query the subset of nodes only and indirectly,without querying other unnecessary nodes. The search algorithm may beconfigured to query the subset of nodes only and directly, withoutquerying other unnecessary edges. The search algorithm may be configuredto query the subset of nodes only and indirectly, without querying otherunnecessary edges. The search algorithm may be configured to query thesubset of nodes only and directly, without querying other unnecessarynodes and edges. The search algorithm may be configured to query thesubset of nodes only and indirectly, without querying other unnecessarynodes and edges. The unnecessary nodes and edges may be the nodes andedges that are not related to the queries.

The FHDN may allow queries at any portion of the network at a historicaltime instance to be answered, while using only about 10 MB to 100 MB ofstorage compared to 2.1 TB using conventional graphing techniques. Theamount of memory required to store the FHDN may be at most 40 MB, 30 MB,25 MB, 20 MB, 19 MB, 18 MB, 17 MB, 16 MB, 15 MB, 14 MB, 13.4 MB, 13 MB,12 MB, 11 MB, 10 MB, 9 MB, 8 MB, 7 MB, 6 MB, 5 MB, 4 MB, 3 MB, 2 MB, 1MB or less. In some cases, the amount of memory required to store theFHDN may be much greater depending on the size, complexity, and age ofthe system that the FHDN represents.

Bill of Materials

The system may comprise the Bill of Materials for any manufacturingcompany. The bill of materials (BOM) may comprise a list of the rawmaterials, sub-assemblies, intermediate assemblies, sub-components,parts, and the quantities of each needed to manufacture an end product.A BOM may be used for communication between manufacturing partners orconfined to a single manufacturing plant.

A BOM can define products as they are designed (engineering bill ofmaterials), as they are ordered (sales bill of materials), as they arebuilt (manufacturing bill of materials), or as they are maintained(service bill of materials). The different types of BOMs depend on thebusiness need and use for which they are intended. The BOM may alsocomprise the formula, recipe, or ingredients list. In electronics, theBOM may represent the list of components used on the printed wiringboard or printed circuit board.

The system may comprise a supply chain distribution network. The supplychain distribution network may comprise a plurality of players (e.g.buyers or sellers). The plurality of players may be connected to eachother through a plurality of connections. The connections may representbuying, selling, or buying and selling.

The state of the supply chain distribution network may comprisegraphical states of the plurality of players at the historical timeinstance. The state of the supply chain distribution network maycomprise graphical states of the plurality of connections connectingplayers at the historical time instance.

The state of the supply chain distribution network may comprisegraphical states of a subset of the players at the historical timeinstance. The graphical state may comprise all the players that areactive in the supply chain distribution network at the historical timeinstance. All the players at the historical time instance may be asubset of the system. The subset of the players may comprise at leastabout: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%,80%, 90% or greater of players of the supply chain distribution network.In other cases, the subset of the players may comprise at most about90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, 15%, 10%, 9%, 8% or less of theplayers of the supply chain distribution network.

The FHDN may the query for a historical time instance to be answeredwithout requiring full network instantiation of the supply chaindistribution network at the historical time instance. In the supplychain distribution network, the FHDN may comprise a plurality of players(represented by a plurality of nodes), a plurality of connectionsconnecting the players (represented by a plurality of edges), and timeseries associated with each of the plurality of players and connections.The FHDN of the supply chain distribution system may comprise all theplayers and all the connections that have ever existed in the supplychain distribution system.

The plurality of nodes and edges and the time series may be associatedwith the plurality of players. The plurality of nodes and edges maycomprise (1) all nodes and edges that had previously existed in thesupply chain distribution system at any given time instant, and (2) allnodes and edges that are presently existing in the supply chaindistribution system. In other embodiments, the plurality of nodes andedges comprises (1) all nodes and edges that had previously existed inthe supply chain distribution system at any given time instant, (2) allnodes and edges that are presently existing in the supply chaindistribution system, and (3) all nodes and edges that will be existingin the supply chain distribution system

The time series may comprise precise times of additions or removals forthe selected node or edge in the supply chain distribution system. Thetime series may comprise precise times of additions, removals, oramendments for the selected node or edge in the supply chaindistribution system. In the supply chain distribution system, theadditions may represent a player is open to buy or sell from otherplayers, the removals may represent a player is not open to buy or sellfrom other players, the amendments may represent a player changeshis/her positions in buying or selling from other players. The timeseries may be obtained through systematic or non-systematic timeintervals. For the non-systematic time intervals, the time series may befirst obtained at a first time for a first time period, and thenobtained at a second time for a second time period. For instance, thetime series may be first obtained at every 1 s for a first time periodof 1 minute, and then obtained every 10 s for a second time period of 10hours.

The graph search algorithm may be utilized to query the states of thesupply chain distribution network at a given time instant, by onlysearching the nodes and edges that are active in the supply chaindistribution network. The graph search algorithms may specify an orderto search through the nodes of a graph. The graph search algorithm maybe, for example, a connected component search such as a depth-firstsearch or breadth-first search. For instance, the graph searchalgorithms may start at the source node and keep searching until thetarget node is found, then the frontier may comprise nodes that haven'tbeen explored yet, and each iteration, a node may be taken off thefrontier, and add its neighbors to the frontier. The as-needed basis canbe set up by the users. For example, if an analytic requires informationabout a small subset of nodes at a given time, the FHDN data structurecan be queried directly to assess the results of this analytic withouttouching any unnecessary nodes. The search algorithm may be anyalgorithm which solves the search problem to retrieve information storedwithin some data structure or calculated in the search space of aproblem domain. Examples search algorithms are described elsewhereherein. The graph search algorithms may specify an order to searchthrough the nodes of a graph. The graph search algorithm may be adepth-first search algorithm, a breadth-first search algorithm,Dijkstra's Algorithm, or the like.

The graph search algorithm may not be configured to query thenon-selected nodes and edges contained in the supply chain distributionnetwork. The search algorithm may be configured to query the subset ofnodes and/or edges only and directly, without querying other unnecessarynodes and/or edges. The search algorithm may be configured to query thesubset of nodes and/or edges only and indirectly, without querying otherunnecessary nodes and/or edges. The unnecessary nodes or edges maycomprise at least about: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%,40%, 50%, 60%, 70%, 80%, 90% or greater of all the nodes or edges,respectively, in the FHDN. In other cases, the unnecessary nodes oredges may comprise at most about 90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%,15%, 10%, 9%, 8% or less of all the nodes or edges, respectively, in theFHDN.

The FHDN may allow queries at any portion of the supply chaindistribution network at a historical time instance to be answered, whilerequiring only small storage. The amount of memory required to store theFHDN may be at most 40 MB, 30 MB, 25 MB, 20 MB, 19 MB, 18 MB, 17 MB, 16MB, 15 MB, 14 MB, 13.4 MB, 13 MB, 12 MB, 11 MB, 10 MB, 9 MB, 8 MB, 7 MB,6 MB, 5 MB, 4 MB, 3 MB, 2 MB, 1 MB or less. In some cases, the amount ofmemory required to store the FHDN may be much greater depending on thesize, complexity, and age of the system that the FHDN represents.

Social Network

The system may comprise a social network comprising of a plurality ofusers. The system may comprise a plurality of social networks. Theplurality of social networks may be connected to each other through aplurality of connections. A given social network of the plurality ofsocial networks may comprise a plurality of users (represented by nodes)and a plurality of connections (represented by edges).

The systems described herein can be used to query the state of socialnetwork at particular times in history. Such states can be used to trackdemographic changes in the population of people on the social network,for example. Such states may also be of interest to users of socialnetworks so that such users can see how their social circles have grownover time. The state of the system may comprise graphical states of theplurality of social networks at the historical time instance. The stateof the system may comprise graphical states of the plurality ofconnections connecting social networks at the historical time instance.The number of social networks at the historical time may be at least 1,10, 50, 100, 200, 300, 400, 500, 1000, 10000 or greater. The number ofsocial networks at the historical time may be at most 10000, 1000, 500,400, 300, 200, 100, 50, 10 or less. The number of connections connectingsocial networks at the historical time instance may be at least 1, 10,50, 100, 200, 300, 400, 500, 1000, 10000 or greater. The number ofconnections connecting social networks at the historical time instancemay be at most 10000, 1000, 500, 400, 300, 200, 100, 50, 10 or less.

The state of the system may comprise graphical states of a subset of thesocial networks at the historical time instance. The graphical state maycomprise all the social networks that are active at the historical timeinstance. All the social networks at the historical time instance may bea subset of the system. The subset of the social networks may compriseat least about: 4%, 5%, 6%, 7%, 8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%,60%, 70%, 80%, 90% or greater of full social networks of the system. Inother cases, the subset of the social networks may comprise at mostabout 90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, 15%, 10%, 9%, 8% or lessof the full social networks of the system.

The FHDN may permit the query for a historical time instance to beanswered without requiring full network instantiation of the system atthe historical time instance. In the system, the FHDN may comprise aplurality of users (represented by a plurality of nodes), a plurality ofconnections connecting the users (represented by a plurality of edges),and time series associated with each of the plurality of users andconnections. The connections may represent following or adding a friend.The FHDN of the social network may comprise all the users and all theconnections that have ever existed in the system.

The plurality of nodes and edges and the time series may be associatedwith the plurality of social networks and the connected nodes andbranches within each social network. The plurality of nodes and edgesmay comprise (1) all nodes and edges that had previously existed in thesocial networks at any given time instant, and (2) all nodes and edgesthat are presently existing in the social networks. In otherembodiments, the of nodes and edges may comprise (1) all nodes and edgesthat had previously existed in the social networks at any given timeinstant, (2) all nodes and edges that are presently existing in thesocial networks, and (3) all nodes and edges that will be existing inthe social networks.

The time series may comprise precise times of additions or removals forthe selected node or edge in the social networks. The time series maycomprise precise times of additions, removals, or amendments for theselected node or edge in the social network. In the social networks, theadditions may represent a user is adding another user as a friend orfollowing another user, the removals may represent a user is blocking orunfriending another user, and the amendments may represent a userchanges his/her positions towards another user. For the systematic timeintervals, the time series may be first obtained at a first time for afirst time period, and then obtained at a second time for a second timeperiod. For instance, the time series may be first obtained at every 1 sfor a first time period of 1 minute, and then obtained every 10 s for asecond time period of 10 hours.

The graph search algorithm may be utilized to query the states of thesocial networks at any given time instant, by only searching the nodesand edges that are contained in the interested social networks. Thesearch algorithm may be any algorithm which solves the search problem toretrieve information stored within some data structure or calculated inthe search space of a problem domain. Examples search algorithms aredescribed elsewhere herein.

The graph search algorithm may be configured to only query the status ofnodes and edges that are contained in the selected social networks on anas-needed basis. The graph search algorithm may not be configured toquery the nodes and edges contained in other non-selected socialnetworks. The search algorithm may be configured to query the subset ofnodes and/or edges only and directly, without querying other unnecessarynodes and/or edges. In other embodiments, the search algorithm may beconfigured to query the subset of nodes and/or edges only andindirectly, without querying other unnecessary nodes and/or edges. Theunnecessary nodes or edges may comprise at least about: 4%, 5%, 6%, 7%,8%, 9%, 10%, 15%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% or greater ofall the nodes and edges, respectively, in the FHDN. In other cases, theunnecessary nodes or edges may comprise at most about 90%, 80%, 70%,60%, 50%, 40%, 30%, 20%, 15%, 10%, 9%, 8% or less of all the nodes andedges, respectively, in the FHDN.

The amount of memory required to store the FHDN may be at most 40 MB, 30MB, 25 MB, 20 MB, 19 MB, 18 MB, 17 MB, 16 MB, 15 MB, 14 MB, 13.4 MB, 13MB, 12 MB, 11 MB, 10 MB, 9 MB, 8 MB, 7 MB, 6 MB, 5 MB, 4 MB, 3 MB, 2 MB,1 MB or less. In some cases, the amount of memory required to store theFHDN may be much greater depending on the size, complexity, and age ofthe system that the FHDN represents.

OTHER EMBODIMENTS

In another aspect, a system for determining a historical state of adynamic network may comprise a data aggregation component and a networkgraphing component.

The data aggregation component may be configured to continuously obtaindata associated with a system from a plurality of different datasources. The data sources are explained elsewhere herein. The dataaggregation component may perform data aggregation. The data aggregationmay comprise graph aggregation. In graph aggregation, the temporalvariation in the topology and parameter values can be represented usingaggregates as edge/node attributes in the graph used to represent thespatial network. The edges and nodes can disappear from the networkduring certain instants of time and new nodes and edges can be added.The time aggregated graph can keep track of these changes through a timeseries attached to each node and edge that indicates their presence atvarious instants of time.

The network graphing component may be configured to construct a fullhistory dynamic network (FHDN) of the system using the data and providea state of the system for a historical time instance in response to aquery of the FHDN for the historical time instance. The FHDN maycomprise (1) a plurality of nodes that are capable of dynamicallychanging, (2) a plurality of edges connecting the nodes, the edges beingcapable of dynamically changing, and (3) time series associated witheach of the plurality of nodes and edges. The FHDN is describedelsewhere herein.

The system may further comprise a display component. The displaycomponent may comprise a speaker or a display screen. The speaker and/ordisplay screen may be operatively coupled with an electronic device. Thespeaker and/or display screen may be integrated with an electronicdevice. The electronic device may be a small alert apparatus. Theelectronic device may be a portable electronic device. The electronicdevice may be mobile phones, PCs, tablets, printers, consumerelectronics, and appliances. The electronic devices may be wearabledevices, including but not limited to, Fitbit, Apple watch, Samsunghealth, Misfit, Xiaomi Mi band, and Microsoft band. The display screenmay be a liquid crystal display, similar to a tablet computer. Thedisplay screen may be accompanied by one or more speakers and may beconfigured to provide visual and audial instructions to a user. Thespeaker may comprise smart speakers. The smart speakers may compriseAlexa, Google Home, Google Assistant, Clova, Microsoft Cortana,AliGenie, Ambient, Apple HomeKit, Apple Siri, and Apple Pod.

In another aspect, a non-transitory computer readable medium may storeinstructions that, when executed by one or more servers, causes the oneor more servers to perform a method comprising continuously obtainingdata associated with a system from a plurality of different datasources; constructing a full history dynamic network (FHDN) of thesystem using the data; and providing a state of the system for ahistorical time instance in response to a query of the FHDN for thehistorical time instance.

The data may be stored in a database. A database can be stored incomputer readable format. A computer processor may be configured toaccess the data stored in the computer readable memory. A computersystem may be used to analyze the data to obtain a result. The resultmay be stored remotely or internally on the non-transitory computerreadable medium and communicated to users of the system or FHDN. Thenon-transitory computer readable medium may be operatively coupled withcomponents for transmitting the result. Components for transmitting caninclude wired and wireless components. Examples of wired communicationcomponents can include a Universal Serial Bus (USB) connection, acoaxial cable connection, an Ethernet cable such as a Cat5 or Cat6cable, a fiber optic cable, or a telephone line. Examples for wirelesscommunication components can include a Wi-Fi receiver, a component foraccessing a mobile data standard such as a 3G or 4G LTE data signal, ora Bluetooth receiver. All these data in the non-transitory computerreadable medium may be collected and archived to build a data warehouse.

The FHDN may comprise (1) a plurality of nodes that are capable ofdynamically changing, (2) a plurality of edges connecting the nodes, theedges being capable of dynamically changing, and (3) time seriesassociated with each of the plurality of nodes and edges. The FHDN isdescribed elsewhere herein.

The systems and methods described herein can be implemented, for exampleusing various embodiments of the platform described in U.S. PatentApplication Publication No. 2018/0191867 (entitled “Systems, Methods,and Devices for an Enterprise AI and Internet-of-Things Platform) whichis herein incorporated by reference in its entirety.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. It is not intendedthat the invention be limited by the specific examples provided withinthe specification. While the invention has been described with referenceto the aforementioned specification, the descriptions and illustrationsof the embodiments herein are not meant to be construed in a limitingsense. Numerous variations, changes, and substitutions will now occur tothose skilled in the art without departing from the invention.Furthermore, it shall be understood that all aspects of the inventionare not limited to the specific depictions, configurations or relativeproportions set forth herein which depend upon a variety of conditionsand variables. It should be understood that various alternatives to theembodiments of the invention described herein may be employed inpracticing the invention. It is therefore contemplated that theinvention shall also cover any such alternatives, modifications,variations or equivalents. It is intended that the following claimsdefine the scope of the invention and that methods and structures withinthe scope of these claims and their equivalents be covered thereby.

1. A computer-implemented method of determining a historical state of asystem, the method comprising: obtaining data about the system from aplurality of different data sources; constructing a full history dynamicnetwork (FHDN) of the system using the data, wherein the FHDN comprises(1) a plurality of nodes that are capable of dynamically changing, (2) aplurality of edges connecting the nodes, wherein the plurality of edgesare capable of dynamically changing, and (3) time series associated witheach of the plurality of nodes and the plurality of edges wherein thetime series indicate changes in state of the plurality of nodes and theplurality of edges over time; and providing a state of the system for ahistorical time instance in response to a query of the FHDN for saidhistorical time instance.
 2. The method of claim 1, wherein the state ofthe system comprises an exact graphical state of the system, in anas-operated state, at the historical time instance.
 3. The method ofclaim 1, wherein the state of the system comprises an exact graphicalstate for a subset of the system at the historical time instance.
 4. Themethod of claim 1, wherein a historical dynamic behavior of the FHDN isdetermined without analyzing a sequence of snapshots of the networkcaptured at different points in time.
 5. The method of claim 1, wherethe FHDN permits the query for the historical time instance to beanswered without requiring full network instantiation at said historicaltime instance.
 6. The method of claim 1, wherein the plurality of nodesand edges comprises (1) all nodes and edges that had previously existedin the network beginning at a given time instance, and (2) all nodes andedges that presently exist in the network.
 7. The method of claim 1,wherein the time series for a selected node or edge comprises times ofadditions or removals of the selected node or edge in the network. 8.The method of claim 1, wherein the time series are based on events orchanges occurring at a selected node or edge.
 9. The method of claim 1,wherein the state of the system at the historical time instance isobtained by using a search algorithm that iterates over the plurality ofnodes and searches through the time series.
 10. The method of claim 9,wherein the search algorithm comprises an iterative graph searchalgorithm that is configured to only check the status of selected nodesor edges on an as-needed basis.
 11. The method of claim 9, wherein thequery comprises an information request about a subset of nodes at agiven time instance, and wherein the search algorithm is configured toquery the subset of nodes only and directly, without querying otherunnecessary nodes.
 12. The method of claim 1, further comprising:utilizing a blocking technique to cache entire connected graphicalregions of the FHDN in memory for any given instance in time.
 13. Themethod of claim 12, wherein the connected graphical regions of the FHDNcontain no unreachable nodes.
 14. The method of claim 12, wherein theblocking technique comprises standard blocking, token blocking, orattribute-clustering blocking.
 15. The method of claim 13, wherein thecaching of the entire connected graphical regions in memory allowssearches to be carried out more quickly compared to conventional networkgraphing techniques.
 16. The method of claim 1, wherein the use of theFHDN allows memory/storage savings of at least three orders of magnitudecompared to conventional network graphing techniques. 17.-29. (canceled)30. The method of claim 1, wherein the FHDN is stored in atwo-dimensional matrix comprising a plurality of rows and columns,wherein each row of the plurality of rows represents one of theplurality of nodes, wherein each column of the plurality of columnsrepresents one of the plurality of edges, and wherein an entry at a rowand a column in the two-dimensional matrix indicates whether the edgerepresented by the column is connected to the node represented by therow.
 31. The method of claim 30, wherein the entry is a time series. 32.The method of claim 1, wherein the FHDN is stored in a graph database.33. The method of claim 32, wherein the graph database comprises aplurality of pointers between the plurality of nodes, wherein eachpointer represents one of the plurality of edges.
 34. The method ofclaim 33, wherein the plurality of pointers comprises a pointer that isbidirectional.
 35. The method of claim 33, wherein the plurality ofpointers comprises a pointer that is unidirectional.
 36. The method ofclaim 1, wherein the plurality of nodes and the plurality of edgescomprise tags or properties.
 37. The method of claim 1, wherein theplurality of edges comprises weights that indicate strength ofconnections or relationships between nodes.
 38. The method of claim 1,wherein the system comprises the Bill of Materials for any manufacturingcompany. 39.-44. (canceled)
 45. The method of claim 1, wherein the FHDNis constructed without requiring periodic capturing and storing ofsnapshots of the system at different points in time.
 46. The method ofclaim 1, wherein constructing the FHDN of the system comprisesgenerating data objects that represent the plurality of nodes, theplurality of edges, and the time series.
 47. A system for determining ahistorical state of a system, the system comprising: a data aggregationcomponent for continuously obtaining data about a system from aplurality of different data sources; and a network graphing componentconfigured to: construct a full history dynamic network (FHDN) of thesystem using the data, wherein the FHDN comprises (1) a plurality ofnodes that are capable of dynamically changing, (2) a plurality of edgesconnecting the nodes, wherein the plurality of edges are capable ofdynamically changing, and (3) time series associated with each of theplurality of nodes and the plurality of edges, wherein the time seriesindicate changes in state of the plurality of nodes and the plurality ofedges over time, and wherein the FHDN is constructed without requiringperiodic capturing and storing of snapshots of the system at differentpoints in time; and provide a state of the system for a historical timeinstance in response to a query of the FHDN for said historical timeinstance.
 48. (canceled)